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DETAILED ACTION 

Reopening Prosecution after Appeal Brief 

1 . In view of the appeal brief filed on 1 1 August 2006, PROSECUTION IS HEREBY 

REOPENED. A new ground of rejection is set forth below. 

2. To avoid abandonment of the application, appellant must exercise one of the following 
two options: 

(1) file a reply under 37 CFR 1 . 1 1 1 (if this Office action is non-final) or a reply under 37 CFR 

1.113 (if this Office action is final); or, 

(2) initiate a new appeal by filing a notice of appeal under 37 CFR 41.31 followed by an appeal 
brief under 37 CFR 41 .37. The previously paid notice of appeal fee and appeal brief fee can be 
applied to the new appeal. If, however, the appeal fees set forth in 37 CFR 41.20 have been 
increased since they were previously paid, then appellant must pay the difference between the 
increased fees and the amomt previously paid. 

A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by signing below. 

Response to Arguments 

3. Applicant's arguments, see Appeal Brief, filed 12/14/2007, with respect to the 
rejection(s) of claim(s) 1-45 under 35 U.S.C. 103(a) have been fiiUy considered and are 
persuasive. Therefore, the rejection has been withdrawn. However, upon fiirther consideration, 
a new ground(s) of rejection is made in view of Aoki et al, Uchihashi et al and Rui et al. 
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Specification 

4. The specification is objected to as failing to provide proper antecedent basis for the 
claimed subject matter. See 37 CFR 1.75(d)(1) and MPEP § 608.01(o). 

Correction of the following is required: Applicant claims a "computer-readable storage 
media" in claims 34-45, which is not defined in the specification. 

Drawings/Specification 

5. The drawings are objected to as failing to comply with 37 CFR 1 .84(p)(5) because they 
do not include the following reference sign(s) mentioned in the description: the description of 
figure 3 in the specification includes an element 13, which is not included in the drawing; and the 
description of figure 5 in the specification includes an element 252, which is not included in the 
drawing; the description of figure 6 includes an element 360, which is not included in the 
drawing but the drawing includes an element 310, which is not described in the specification. 
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office 
action to avoid abandonment of the application. Any amended replacement drawing sheet should 
include all of the figures appearing on the immediate prior version of the sheet, even if only one 
figure is being amended. Each drawing sheet submitted after the filing date of an appUcation 
must be labeled in the top margin as either "Replacement Sheet" or "New Sheet" pursuant to 37 
CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and 
informed of any required corrective action in the next Office action. The objection to the 
drawings will not be held in abeyance. 
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Applicant is required to amend the specification as needed to include any drawing 
corrections. 

Claim Objections 

6. Claim 9 is objected to because of the following informalities: "the communication 
interactions include a communication interaction pertains to. . ." should be changed to —the 
communication interactions include a communication interaction that pertains to--. Correction is 
required. 

Claim Rejections - 35 USC § 103 

7. The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 1 02 ol' this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

8. This application currently names joint inventors. In considering patentability of the 
claims under 35 U.S.C. 103(a), the examiner presumes that the subject matter of the various 
claims was commonly owned at the time any inventions covered therein were made absent any 
evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out 
the inventor and invention dates of each claim that was not commonly owned at the time a later 
invention was made in order for the examiner to consider the applicability of 35 U.S.C. 103(c) 
and potential 35 U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 
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9. Claims 1-8, 14-29, 32-41, 44 and 45 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Aoki et al (US 2004/0172255 Al) and Uchihashi et al (US 2004/0201710 
Al). 

Claims 1 and 22: 

Aolii discloses a system for communication (abstract; p. 1, par. 001 1). Aoki discloses a 
first set of sensing and rendering components arranged to cover physical movements of multiple 
individuals present in a first environment. Aoki teaches a plurality of sensing components, such 
as microphones (p. 3, par. 0047) and vocalization detectors (p. 3, par. 0047); and rendering 
components, such as visual displays (p. 12, par. 0157; fig. 1), headphones (p. 14, par. 0170, lines 
1-1 1), and audio speakers (p. 3, par. 0046), to cover physical activities and/or presence (tracking 
users' physical locations: p. 13, par. 0168, lines 8-10) of the individuals present in the 
environment. Aoki explains that the sensing and rendering components are used for detecting 
conversational characteristics (p. 4-5, par. 0061), such as user's physiological reactions (p. 9, par. 
01 14), user's vocalizations (p. 9, par. 01 15), user's physical activity (p. 13, par. 0168), gestures 
such as mouse-clicks, button pushes, or voice commands directed at a computer (p. 14, par. 
0174) of multiple individuals present in a (first environment) conversational floor (p. 1, par. 
0012, 0013). Aoki teaches a context-awareness system that may be combined with many kinds 
of physical sensor data and computer application data to make assessments of user activity; the 
system being capable of tracking user's physical locations within a building and compute which 
users are co-present in a room (motion or presence detection), (p. 13-14, par. 0168). 

Aoki discloses a second set of sensing and rendering components arranged to cover 
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physical movements of multiple individuals in a second environment. Aoki teaches a system that 
enables groups and subgroups of people to communicate and carry on separate conversations 
within the context of a meeting (p. 1-2, par. 001 1-0014). Aoki explains that the invention can 
include additional types of inferential analysis that use input data other than those directly 
affected by conversation. The system may combine many kinds of physical sensor data and 
computer application data (p. 14, par. 0170) to make assessment of user activity for users who 
are co-present in a room and/or users who are not present (first and second environments) in the 
room (p. 13, par. 0168-0169). 

Aoki discloses interest thread detector that uses the first and second set of sensing and 
rendering components to detect multiple communication interactions each involving a respective 
subset of the individuals present in the first and second environments and that maintains an 
interest thread for each communication interaction. Aoki teaches detecting multiple 
communication interactions involving subgroups of individuals present in the same or in 
different environments (p. 13-14, par. 0168-0169). Aoki teaches maintaining an interest thread 
for each communication interaction (p. 14, par. 0170). Aoki explains that the invention has a 
floor analysis module used to analyze data for conversational characteristics and that its results 
are used to determine user's activities, specify a set of mixing parameters, and communicate the 
mixing parameters over a floor control path (p. 4, par. 0056, 0061; p. 5, par. 0061-0062; p. 6, par. 
0072, 0073, 0076). 

Aoki discloses communication provider that captures a set of media data from the 
sensing components and that combines the captured media data in response to the respective 
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activities of each subset of the individuals and that communicates the combined media data to 
the rendering components. Aoki teaches a communication system and a method for receiving a 
plurality of communications from a plurality of communication sources; for mixing the plurality 
of communication for a plurality of outputs associated with the plurality of communication 
sources (p. 1, par. 001 1-0013; p. 2, par. 0014-0015). Aoki teaches combining the captured media 
data and communicating the combined media data to the rendering components (p. 14, par. 0173- 
0178). Aoki teaches a plurality of modules and a "conversational floor configuration thread" for 
analyzing conversational characteristics of the plurality of individuals (p. 4, par. 0056-0058, 
0061, 0062), and using the resuks of the analysis to control the floor controls and/or set of 
mixing parameters. Aoki explains that the floor conflguration thread is invoked by the "invoke 
floor conflguration thread" and that this thread is responsible for receiving results of the 
individual analysis modules, determining and selecting the conflguration of audio sources from 
these results (p. 6, par. 0076). 

Aoki teaches analyzing data for conversational characteristics that can be determined 
from analysis of audio information or from physiological responses to the conversation, such as 
responses measured by a biometric device and/or information from an eye-tracker device (par. 
0114) but Aoki does not specifically teach detecting physical movement of the individuals. 
However, Uchihashi discloses a system and method for computer assisted recording or capture 
of meetings or presentation events (abstract; p. 1, par. 0008). Uchihashi teaches a system having 
multiple cameras and sensors (p. 1, par. 0017, 0019) for detecting physical motion of humans 
present in a conference (p. 2, par. 0022-0024, 0029; p. 4, par. 0046-0047). Uchihashi teaches 
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displaying "candidate activity events", which are events of potential interest (interest thread) in 
the meeting (p. 5, par. 005 1). Therefore, it would have been obvious to one ordinarily skilled in 
the art at the time the invention was made to modify Aoki's method of determining and 
analyzing the activities of multiple participants in a meeting to include Uchihashi's teaching of 
using sensors to capture physical movements of the participants because as Uchihashi says, the 
system follows the flow of the individuals' and/or the presentation's activities within the 
conference or meeting, and also because delays caused by camera switching latency is reduced 
or eliminated when the system captures only those activities that are of real interest to the 
meeting. 

Claims 2. 23 and 35: 

See claim 1 . Aoki teaches selecting respective subsets of the first and second set of 
sensing and rendering components for use for each communication interaction (interest thread) 
or user's activities. Aoki explains that a meeting can have subgroups of people who carry on 
separate conversations, and that each subgroup can maintain awareness of the primary group 
conversation (p. 2, par. 0038). Aoki teaches a conversational floor wherein users are enabled to 
have side conversations in the conversational environment (p. 6-7, par. 0081-0082; p. 9-10, par. 
0120). 

Claims 3. 24 and 36: 

Aoki teaches activities including speech levels of the individuals (user's vocalization) 
involved in the communication interaction (p. 3, par. 0035; p. 9, par. 01 15, 0118). 
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Claims 4. 25 and 37: 

Uchihashi teaches activities including gestures by the individuals involved in the 
communication interaction (p. 3, par. 0035). 
Claims 5. 18.20. 26 and 38: 

Uchihashi teaches activities including movements by the individual involved in a 
respective interest thread (p. 2, par. 0022-0024, 0029; p. 4, par. 0046-0047). 
Claims 6. 27 and 39: 

Uchihashi teaches capturing and storing location information about identified objects and 
individuals within the conference room (p. 3, par. 0031; p. 4, par. 0043). 
Claims 7. 28 and 40: 

Aoki teaches refining captured data (p. 3, par. 0048). Uchihashi teaches processing the 
raw sensor information (p. 2, par. 0021; p. 4, par. 0045; p. 5, par. 0051). 
Claims 8. 29 and 41: 

Aoki/Uchihashi teaches storing captured data (Aoki: par. 0047; Uchihashi: par. 0022, 
0031,0033). 

Claims 14-15. 32-33 and 44-45: 

See claim 1 . Aoki teaches capturing communication interactions (interest threads) of 
conversational activities of different groups of individuals (main conversation and side 
conversation) creating in this way interest area for separate detected activities (p. 6-7, par. 0081; 
p. 9-10, par. 0120, 0130; p. 13, par. 0168). 
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Claims 16 and 17: 

See claim 1 . Aoki teaches communication interactions involving two or more individuals 
(p. 1, par. 0011) in two or more environments (p. 13-14, par. 0168-0169). 

Claim 19: 

Aoki/Uchihashi teaches movement pertaining to rendering devices (Uchihashi: p. 2, par. 
0020; p. 3, par. 0035). 
Claim 21: 

See claim 1 . Aoki teaches participants may be in the same room or at a remote location 
(p. 13-14, par. 0168, 0170). 
Claim 34: 

See claim 1. Aoki discloses a computer-readable media that containing code (p. 4, par. 
0050, lines 1-4). 

10. Claims 9-13, 30, 31, 42 and 43 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Aoki/Uchihashi and Rui et al (US 2002/0191071 Al). 

Claims 9-13.30,31,42 and 43: 

Aoki/Uchihashi teaches tracking interactions pertaining to an individual or an object (p. 
2, par. 0020, 0031) but it does not teach communication interactions pertaining to an artifact that 
changes over time. However, Rui discloses a method for recording and broadcasting meetings 
(abstract; p. 1, par. 0009) wherein communication interactions that pertain to an artifact, such as 
a whiteboard, are captured and stored (p. 1-2, par. 0010). Thus, it would have been obvious to a 
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person having ordinary skill in the art at the time the invention was made to modify 
Aoki/Uchihashi's method of capturing communication interactions to include Rui's teaching of 
capturing communication interactions that pertain to devices because the participants are enabled 

to share a workspace, such as a whiteboard, and capture annotations that can be viewed by all 
participants in the conference room and then later reviewed by those individuals who were or 
were not present in the meeting. 

Conclusion 

1 1 . The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

12. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to X. L. Bautista whose telephone number is (571) 272-4132. The 
examiner can normally be reached on Monday-Thursday 8:00AM-6:00PM. 

If attempts to reach the examiner by telephone are unsuccessfiil, the examiner's supervisor, 
Weilun Lo can be reached on (571) 272-4847. The fax phone number for the organization where 
this application or proceeding is assigned is 571-273-8300. 

1 3 . Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
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system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/X. L. Bautista/ 

Primary Examiner, Art Unit 2179 



/Weilun Lo/ 

Supervisory Patent Examiner, Art Unit 2179 
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February 13, 2008 



